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Lack of researcher consensus on how to measure disadvantaged students’ access to effective teaching has 
made it challenging for practitioners to draw lessons from the data. This brief aims to help policymakers 
understand the emerging evidence by synthesizing findings from three peer-reviewed studies that collec- 
tively span 1 7 states. The studies provide two lessons: (1) on average, disadvantaged students received 
less effective teaching than other students, equivalent to about four weeks of learning for reading and 
two weeks for math, or about 2 to 4 percent of the student achievement gap between these groups; and 
(2) access to effective teaching for disadvantaged students varied across districts, with a statistically 
significant difference between more and less disadvantaged students ’access in some districts and no 
statistically significant difference in access in others. 


Recent federal initiatives are designed, in part, to improve disadvantaged students’ access to effective 
teaching. Under the Elementary and Secondary Education Act (ESEA), states must ensure that poor and 
minority students are taught by qualified teachers at similar rates as other students. The U.S. Department 
of Education’s (ED’s) ESEA flexibility policy maintains this equity requirement by encouraging states 
to assess whether disadvantaged students have equal access to effective teachers, as measured by teacher 
evaluation systems that use student achievement growth and classroom observations (U.S. Department of 
Education 2012). Assisting this effort are two federal grant programs — Race to the Top and the Teacher 
Incentive Fund — which support state and local policy initiatives designed to improve disadvantaged 
students’ access to effective teachers (U.S. Department of Education 2009, 2010). 

As federal policy focuses on improving access to effective teaching, there is a need to better understand 
how effective teachers are distributed across schools and districts. To what extent do disadvantaged stu- 
dents receive less effective teaching than other students? If so, how much less effective are teachers of 
disadvantaged students? 

ED’s Institute of Education Sciences (IES) sponsored three studies that address these questions. The 
findings are synthesized in this brief. In two of the studies, Sass et al. (2012) and Isenberg et al. (2013) 
explored whether disadvantaged students received less effective teaching than other students. 1 In the third 
study, Glazerman and Max (2011) measured whether the best teachers were under-represented in schools 
with more disadvantaged students. All three studies defined effective teaching based on value-added mea- 
sures. “Value added” is a teacher’s contribution to students’ learning gains, often assessed based on several 
years of test scores. The studies (and this brief) define “disadvantaged” students as those with family 
income low enough to qualify for free or reduced-price lunches (FRL). The key findings are as follows: 

• Disadvantaged students received less effective teaching on average. The Isenberg et al. study of 
29 districts showed that disadvantaged students received less effective teaching than other students in 
grades 4 through 8 in those districts. Sass et al. reached the same conclusion in grades 4 and 5 for the 
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two states they examined. The average disparity in teaching effectiveness was equivalent to about four 
weeks of learning for reading and two weeks for math. 2 For context, the overall achievement gap for 
disadvantaged students in grades 4 through 8 is equivalent to about 24 months in reading and 1 8 months 
of math, so the differences in teaching effectiveness for one year represent 4 percent of the existing gap 
in reading and 2 to 3 percent in math. 3 

• Access to effective teaching varied across districts. The size of the differences in effective teaching 
between disadvantaged and non-disadvantaged students varied across the 29 districts studied by Isenberg 
et al. The disparities for each district ranged from no statistically significant difference 4 to a difference 
equivalent to 13 weeks of learning in reading and math in grades 4 through 8. Access to effective teach- 
ing also varied across districts when it was measured as the prevalence of highest-performing teachers in 
the school (Glazerman and Max 2011). In one district, the highest-performing elementary teachers were 
marginally over-represented in schools with the highest percentage of disadvantaged students, while at 
the other extreme, the highest-performing math teachers in another district were six times more preva- 
lent in schools with the lowest percentage of disadvantaged students. 

Emerging Evidence on Access to Effective Teaching 

Despite mounting policy interest, there has been limited research on the extent to which disadvantaged stu- 
dents receive less effective teaching than other students. Past studies examined access to qualified teachers 
based on experience or degrees, but a growing body of evidence finds that such teacher qualifications are 
not strongly associated with teacher effectiveness, as measured by the learning gains that teachers achieve 
with their students (Rivkin et al. 2005; Kane et al. 2006; Gordon et al. 2006; Aaronson et al. 2007; Koedel 
and Betts 2007; Buddin and Zamarro 2008). Even existing studies that do measure access to effective 
teaching using learning gains do so in different ways, making it difficult to synthesize the lessons learned. 

This brief focuses on IES-funded studies that have been peer reviewed. Because this literature is newly 
emerging, with most findings not yet published, we did not conduct a systematic review using traditional 
search engines. We relied instead on targeted searches of relevant sources, such as the websites of several 
organizations that conduct value-added analysis for school systems, a review of all the literature cited in 
existing studies, searches of newspaper archives from cities where teacher value-added information was 
released to the public, as well as conversations with more than a dozen researchers or funders of research 
in the field and those who have access to the value-added data that could be used for research of this nature. 

In addition to the IES-funded studies that are the subject of this brief, other studies on access to effective 
teaching have been conducted by the Tennessee Department of Education (2007), the RAND Corporation 
(Steele et al. 2010), a Cornell University researcher (Mansfield 2012), Education Trust-West (Hahnel and 
Jackson 2012), and Students First New York (2013). Additional evidence comes from newspapers in New 
York City and Washington, DC, which have published maps and graphs showing the distribution of highly 
rated teachers across more- and less-disadvantaged neighborhoods. 5 To date, only the IES-funded studies 
have been peer-reviewed. However, these other studies are discussed in Section A in the appendix. Taken 
together, they do not conflict with the findings from the three peer-reviewed studies summarized here. 
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How Access to Effective Teaching is Measured 

The three studies we synthesize here examine the same phenomenon, access to effective teaching, using 
slightly different approaches (Table 1). In two of the studies, the central question is “Do disadvantaged students 
have lower-perfonning teachers on average than other students?” One of these studies (Isenberg et al. 2013) 
was conducted by Mathematica Policy Research and measures access to effective teaching within 29 districts 
in 16 states. The other study (Sass et al. 2012), conducted by the National Center for Analysis of Longitudinal 
Data in Education Research (CALDER), measured access within 2 states. In the third study, Glazerman and 
Max (2011), the central question is “Are the highest-performing teachers under-represented in schools with 
more disadvantaged students?” This study, also by Mathematica, includes 10 districts in 7 states. As mentioned 
earlier, all three define “disadvantaged” as having low enough family income to qualify for FRL. They also 
measure teacher effectiveness using value-added indicators, which estimate a teacher’s contribution to student 
achievement growth from one year to the next using standardized state assessment scores. 


Table 1. Three Studies on Access to Effective Teaching 


Aspect of 
the Study 

1. Isenberg et al. 
(2013) 

2. Sass et al. 
(2012) 

3. Glazerman and Max 
(2011) 

Number of states 
and districts 

29 districts in 16 states 

2 states (all districts) 

10 districts in 7 states 3 

Subjects 

Math and English/ 
language arts 

Math and English/ 
language arts 

Math and English/ 
language arts 

Grade levels 

4 through 8 

4 and 5 

3 through 8 

Time period 

2008-09 through 201 0-1 1 b 

2000-01 through 2004-05 

2007-08 

Measure of 
effective teaching 

Value-added score based 
on one year of value added 6 

Value-added score based on 
four years of value added data 

Whether teacher has 
value-added score in top 
20 percent for the district; 
value added based on two to 
four years of value-added data 

Measure of access 
to effective teaching 

Compare average teacher 
effectiveness for FRL and 
non-FRL students 

Compare average teacher 
effectiveness for schools 
with higher and lower FRL 
rates 6 

Compare the percentage of 
highest-performing teachers 
in schools with the highest 
and lowest FRL rates 

Main finding 

FRL students receive less 
effective teaching on average 
than non-FRL students for 
math and reading, grades 
4 through 8 

Differences in effective 
teaching between FRL and 
non-FRL students in a given 
year are relatively small as 
a percentage of the student 
achievement gap 

Elementary schools with 
higher FRL rates have 
less effective teaching 
than elementary schools 
with lower FRL rates 

The highest-performing 
teachers within districts are 
under-represented in middle 
schools with the highest FRL 
rates for English/language 
arts and math; the evidence 
for this pattern is weaker for 
elementary schools 


3 Six of the districts in Glazerman and Max (2011) are also included in Isenberg et al. (2013), resulting in 33 unique districts across the two studies. 
b The study focused on the 2007-08 through 2009-10 school years for five of the districts. 

c Although Sass et al. (2012) conducted the analysis at the school level, we used information from the article and the authors to generate 
results at the student level. 
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Value-added indicators, increasingly promoted by policy (for example, U.S. Department of Education 
2012; Tennessee Department of Education 2013; Hillsborough County Public Schools 2011), do have 
limitations. Because they rely exclusively on student test scores as an outcome measure, they are not 
meant to capture all aspects of a teacher’s performance, and they can only be estimated for teachers whose 
students take standardized tests. They tell us about teachers’ average impact on their students’ test scores 
after accounting for students’ background and prior achievement. But value-added indicators assume 
that a teacher has the same impact on all of his or her students. There may be differences in how teachers 
devote their time to different students within the classroom that are not captured by the studies we describe 
here. Also, there may be unmeasured influences, such as the sorting of students across classrooms, that 
value-added indicators fail to account for (Rothstein 2009). Despite their limitations, however, value-added 
indicators have been shown to predict teachers’ future performance (Kane and Staiger 2008; Kane et al. 2013) 
and long-term student outcomes (Chetty et al. 2011). 

In order to view these studies in comparable terms, we present the findings from Sass et al. using an 
approach that is analogous to Isenberg et al. Sass et al. focused on whether schools with mostly disadvan- 
taged students had lower-performing teachers on average than other schools. We converted their findings 
to student-level results, measuring whether disadvantaged students had less effective teaching on average 
than other students (see Section B in the appendix for details). However, because we did not have sufficient 
information to perform the analogous calculations from the Glazerman and Max study, we present those 
results separately. 

The districts in these studies were not selected to be nationally representative. The two Mathematica 
studies included districts that tend to be larger, more urban, and more impoverished than the average U.S. 
district. Sass et al. analyzed access to effective teaching in Florida and North Carolina. Even so, these three 
studies collectively present evidence across a large number of districts that include all four Census regions 
of the United States. 

Finding 1: Disadvantaged Students Receive Less Effective Teaching on Average 

On average, disadvantaged students received less effective teaching than all other students. This was the 
case when comparing effective teaching for FRL versus non-FRL students in grades 4 through 8 within 
29 districts in Isenberg et al. (2013) and in grades 4 and 5 within two states in Sass et al. (2012). The 
differences in effectiveness ranged from 2 to 4 percent of a standard deviation of student achievement. 7 
We translated these findings into average “weeks of learning” to make them more meaningful to educators; 
the results suggest that FRL students, on average, will fall behind non-FRL students by four weeks of 
learning in reading and two weeks in math in a given year, solely because of the quality of their teachers. 8 
These findings are shown in Figure 1. 

Although the two studies showed that FRL students received less effective teaching than non-FRL students, 
the size of the differences in a given year was small compared with the overall gap in student achievement 
between FRL and non-FRL students. This gap is equivalent to 24 months in reading and 18 months in 
math, based on data from Isenberg et al. for grades 4 through 8. For Florida and North Carolina, the states 
studied by Sass et al., the gap was 23 months in math and 19 months in reading in the 4th grade. 9 Thus, the 
differences in effective teaching between FRL and non-FRL students shown in Figure 1 represent 4 percent 
of the achievement gap in reading scores and 2 to 3 percent of the gap in math scores. 
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Figure 1. Disadvantaged Students Receive Less Effective Teaching 


Average weeks of learning produced by teachers of FRL students versus teachers of non-FRL students. 


Isenberg et al. Sass et al. Isenberg et al. Sass et al. 



Source: Isenberg et al. (2013) and calculations based on Sass et al. (2012). See Section C in the appendix for details about how we translated 
the study results into the “weeks of learning” measure. 

Note: Isenberg et al. (2013) includes grades 4-8; Sass et al. (2012) includes grades 4 and 5. 

* Differences between teachers of non-FRL students and FRL students are statistically significant at the 0.05 level. 


In the third study, Glazerman and Max (2011) tested whether the highest-perfonning teachers in grades 
3 through 8 were under-represented in the highest poverty schools within ten districts. 10 The highest-per- 
forming teachers were defined as those having value-added scores in the top 20 percent for their district. 
The researchers grouped schools in each district into five equal-sized categories based on their poverty 
level and examined whether the percentage of highest-perfonning teachers differed across the five catego- 
ries. Equal access (no under- or over-representation) would mean that these teachers made up 20 percent 
of teachers in both the highest and lowest poverty schools, as shown by the dashed black line in Figure 2. 

The study revealed unequal access, on average, to the highest-performing teachers in middle school math 
and English/language arts. The highest-performing teachers made up 15 percent of teachers in the high- 
est-poverty schools versus 29 percent in the lowest-poverty schools for middle school math, meaning that 
students in the lowest poverty schools were almost twice as likely to get top teachers as their counterparts 
in the highest-poverty schools. For middle school English/language arts, the highest-performing teachers 
made up 12 percent of teachers in the highest-poverty schools, compared with 32 percent of teachers in the 
lowest-poverty schools. In other words, students in the lowest-poverty schools were two and a half times 
more likely to have a top teacher than those in the highest-poverty schools. Access to the highest-perfonning 
teachers did not statistically differ at the elementary school level. 
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Figure 2. Prevalence of Highest-Performing Teachers in the Highest- and Lowest- Poverty Schools 


Percentage of teachers who are highest performing. 


40 -| 


32 



Elementary Schools Middle School Middle School 

(Reading and Math) English/Language Arts* Math* 


| Highest-Poverty Lowest-Poverty 

Schools Schools 


Source: Glazemian and Max (2011). 

Note: Schools were divided into five equal-sized categories based on the percentage of FRL students, and the percentage of highest-per- 
forming teachers was compared across the categories. This graph shows the percentage of highest-performing teachers in the lowest-poverty 
category — the 20 percent of schools in each district with the lowest percentage of FRL students — and the highest-poverty category — the 20 
percent of schools in each district with the highest percentage of FRL students. The analysis of elementary schools combined math and reading 
because teachers were responsible for both subjects. 

* Difference in the prevalence of highest perforating teachers between the highest- and lowest-poverty schools is statistically significant at the 0.05 level. 


Finding 2: Disparities in Access to Effective Teaching Vary Across Districts 

Estimated differences in effective teaching for FRL versus non-FRL students varied from district to district. 
In the 29 districts studied by Isenberg et al. (2013), these differences ranged from less than one week of 
learning in reading and math to 13 weeks of learning in reading and 8 weeks of learning in math. 11 This is 
shown in Figure 3, which represents each district with a separate bar. A hollow bar means that differences 
in effective teaching received by FRL and non-FRL students (represented by the height of the bar) were 
not statistically significant; a solid bar means that the differences were statistically significant. The top 
panel shows the difference in effective teaching for reading scores (linked to English/language arts teachers), 
and the bottom panel shows math scores. 

Among the 29 districts, Isenberg et al. found no statistically significant difference in effective teaching 
for FRL versus non-FRL students in 2 districts for reading and in 10 districts for math (Figure 3). In all 
other districts, FRL students received less effective teaching than non-FRL students, but the magnitude 
of these differences varied. However, there were no districts where FRL students received significantly 
more effective teaching than non-FRL students. 
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Access to effective teaching also varied from district to district in the study by Glazerman and Max (2011) 
For example, in one district, students in the lowest-poverty middle schools were almost eight times as likely 
to get top teachers as students in the highest-poverty middle schools for English/language arts (Figure 4, 
top panel, District A). In other districts, differences in the share of highest-performing teachers in the 
middle schools with the most and least poverty were not statistically significant (Figure 4, top panel, 
Districts F, G, E, and H). The results for middle school math teachers (Figure 4, bottom panel) also varied 
from district to district. We do not provide results on variability from Sass et al. (2012) here because they 
did not measure differences in effective teaching separately for each district. 

The studies reviewed here represent a first step in exploring the relationship between student disadvantage 
and access to effective teaching. Although these studies provide evidence of generally unequal access, 
an important early lesson is that conditions vary from district to district. 

Figure 3. Disparities in Access to Effective Teaching Varied Across Districts 


Average weeks of learning that teachers of FRL students produce relative to teachers of non-FRL students. 




Districts 


Source: Isenberg et al. (2013) — grades 4 through 8. 

Notes: A solid bar indicates that differences between teachers of non-FRL students and FRL students are statistically significant at the 0.05 level. A hollow 
bar indicates that the differences are not significant. 
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Figure 4. Disparities in Access to Highest-Performing Teachers Varied Across Districts 


Percentage of middle school English/language arts teachers who are highest performing. 
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Percentage of middle school math teachers who are highest performing. 
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Source: Glazerman and Max (201 1), Figures A.8 (top panel) and A.7 (bottom panel). 

Notes: Districts are identified by letter code only. Highest-poverty schools are those whose percentage of FRL students places them in the top 20 percent 
for the district. Lowest-poverty schools are those whose percentage of FRL students places them in the bottom 20 percent. 

* Difference in the prevalence of highest-performing teachers between the highest- and lowest-poverty schools is statistically significant at the 0.05 level. 
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Endnotes 

1 Throughout this document, we refer to “effective teaching” rather than “effective teachers” to be consistent with the source 
documents reviewed. Isenberg et al. 2013 state on page 6, “From a student’s perspective, what matters is the effectiveness 
of the teaching he or she experiences, whatever the source,” including teacher or school inputs. 

2 Table C.l in the Technical Appendix describes these results in terms of standard deviations of student achievement. 

3 The districts in Isenberg et al. (2013) had a student achievement gap equivalent to 24 months in reading and 18 months in math 
for grades 4 through 8. The two states in Sass et al. (2012) had a student achievement gap equivalent to 23 months in reading 
and 19 months in math in the 4th grade. 

4 A difference is only considered statistically significant if the p-value (probability of observing the value when the true differ- 
ence is zero) is less than 5 percent. 

5 “ ‘Highly Effective’ Teachers are Spread Unevenly Across District,” by Bill Turque, Washington Post, Saturday, November 13, 2010; 
“Teacher Quality Widely Diffused, Ratings Indicate,” by Fernanda Santos and Robert Gebeloff, New York Times, February 24, 2012. 

6 As noted in Isenberg et al. (2013), the authors used a single year of value added because the study examined how access to 
effective teaching changed from year to year. Although researchers recommend using multiple years of data to estimate the 
permanent component of teacher effectiveness, this approach may provide biased estimates of effective teaching in a given 
year if there are true changes in effective teaching from year to year. As a result, there is a trade-off between obtaining unbiased 
estimates of effective teaching in a given year and increasing the precision of individual teachers’ value-added estimates. Given 
that the authors averaged value-added estimates for multiple teachers when measuring access to effective teaching, the precision 
gained by using multiple years of data was less valuable for the study than in other contexts. 

7 We calculated these values using results from Sass et al. (2012) to allow an apples-to-apples comparison with the results presented by 
Isenberg et al. (2013). Sass et al. reviewed our calculations for accuracy. The calculations are described in Section B of the appendix. 

8 To translate a standard deviation (calculated relative to the entire population in each student’s home state) into weeks or months 
of learning, we drew on published estimates by Hill et al. (2008) of the number of standard deviations of growth a student 
makes in a year on a typical test. Hill and colleagues used the national norming sample from seven standardized tests that were 
vertically scaled, meaning that students in consecutive grades were scored on the same scale with the same units. An important 
caveat is that these seven standardized tests may cover different content than the state assessments that were used in the studies 
summarized by this brief. Given that the estimates from Hill et al. (2008) are based on seven standardized tests, they provide the 
best available information for converting the results into weeks of learning. We assumed a nine-month year, with 4.33 weeks per 
month. The studies examined the average amount by which FRL students would fall behind non-FRL students in a given year. 
The studies did not show how much a cohort of students would fall behind over several years. 

9 We calculated achievement gaps for North Carolina and Florida based on data from the National Assessment of Educational 
Progress for 4th graders in 2005. 

10 This study focused on the highest-performing teachers only. To translate differences in the prevalence of highest-performing 
teachers into differences in effective teaching across all teachers, one would have to assume that a school with greater access 
to highest-performing teachers also has correspondingly lower representation from the lowest-performing teachers. 

11 Significance tests confirm that in both subjects, variation in access to effective teaching across study districts is greater than 
would be expected to occur by chance. 
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For more information on the two reports, please visit: 

http://ies.ed.gov/ncee/pubs/20 14400 1 
http://ies.ed.gov/ncee/pubs/20114016 

and 

http://www.sciencedirect.com/science/article/pii/S0094 1190120002 1 6 


This brief was prepared for NCEE by Jeffrey Max and Steven Glazerman of Mathematica Policy Research 
under contract number ED-04-CO-01 12/0007, Project Officer, Elizabeth Warner. 
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